On Characterizing and Assessing the Validity of Behavioral Models and Their Predictions

We present a conceptual framework for the validation of behavioral models and the prediction information derived from them. The setting for this work is a modern product development environment in which design is performed by teams of specialists who exchange information and knowledge. This setting makes validation responsibilities ambiguous and separates users from knowledge relevant to validation. To alleviate these problems, we identify three complementary validation responsibilities—validity characterization, compatibility assessment and adequacy assessment—that together solve the validation problem. We define the responsibilities in terms of formal descriptions of models and predictions that provide accuracy assurances within a welldefined context. Because behavioral models are similar to scientific theories and are a form of knowledge, it is possible to draw upon the philosophy literature to gain insight into validation. We review the relevant epistemology and the philosophy of science literature and identify several conclusions that apply to validation. These conclusions provide perspective on the limitations of the described framework. Although the framework is not a complete solution to the validation problem, it serves as is a conceptual roadmap to understanding and solving the problem. As such, this work raises many fundamental questions about validation and represents a starting point for future investigation.